Title: Tech WTF: Updating on pip and Cargo
Date: 2018-02-04
Modified: 2018-06-24
Category: Blog
Slug: tech-wtf-updating-on-pip-and-cargo
Tags: tech-wtf
Summary: Apparently updating things on pip and Cargo is Nintendo Hard

EDIT 2: Having investigated _even further_, it seems like Pip is even less awful
than I suspected, although it would be _awesome_ if its ``--help`` option for
specific commands was better-documented. Furthermore, ``cargo install-update``
actually has an ``--all`` option! Yet another rewrite! The (rather
hyperbolic) title will remain, but never let it be said I don't admit my
mistakes (four months later).

EDIT: Having investigated all these things further, they're not _quite_ as bad
as I'd thought. In particular, Pip is a bit friendlier than I'd assumed, and
both Pip and Cargo have various helpers that can make this easier. Thus, I've
rewritten everything to take this into account. Also, to be constructive, I've
decided to add some helper scripts of my own for people to use to make their own
interactions with these tools easier.

I recently switched to [Gentoo][1]. There are a bunch of reasons for this,
including, but not limited to:

- systemd being too wtf for me
- the Arch community being unpleasant

Thanks to Gentoo using OpenRC and having one of _the_ best communities ever,
I don't have these problems anymore, and will continue migrating all my devices 
to Gentoo over time. As part of that, I finally decided to properly manage my 
non-system packages. I use a bunch of these, including, but not limited to:

- [Tectonic][2]
- [Pelican][3]
- [youtube-dl][4]
- [Asciidoctor][5]
- [proselint][6]

These are written in several languages: Rust, Python and Ruby, to be exact. Each
of these is managed using a different tool: ``cargo`` for Rust, ``pip`` for
Python and ``gem`` for Ruby. Overall, _installing_ stuff with all of them is
pretty straightforward, and tends to go without issue.

However, when it comes to _updating_ stuff you've installed with these, only ``gem``
really makes it convenient; you just go

```
gem update
```

and you're golden. ``pip`` and ``cargo``, however, make this needlessly harder.
This post will explain how to get simple, one-command updates (or as close to
that as possible) for both ``pip`` and ``cargo``. 

## Updating using pip ##

After spending some time understanding that ``pip install --help`` is where you
should start looking, you'll find the following useful option:

```
 -U, --upgrade               Upgrade all specified packages to the newest available version. The handling of dependencies depends on the
                             upgrade-strategy used.
```

Unfortunately, you still need to tell ``pip`` precisely which packages to update. If
you have only a few, you can probably just memorize or script them, but this shouldn't be
necessary in this day and age. Luckily, there _is_ a package that can solve this
problem: ``pip-autoremove``. In particular, we're most interested in its ``-L`` 
flag:

```
  -L, --leaves  list leaves (packages which are not used by any others).
```

This means that if you use ``pip`` for managing executables, you can now use
``pip-autoremove -L`` to find them, since they typically aren't dependencies.
Unfortunately, ``pip-autoremove`` doesn't take into account whether the Python
package is installed by your package manager or ``pip``, so you'll end up getting a
list of both. Once again, ``grep`` to the rescue:

```
[koz@Sebastian ~]$ pip-autoremove -L | grep '\.local'
```

This will only show those packages which are installed locally. We can
then combine this with an incantation of ``pip install --upgrade --user`` to update everything in one shot:

```
pip install --user --upgrade $(pip-autoremove -L | grep '\.local' | cut -d' ' -f1 | xargs)
```

The use of ``cut`` above limits the output to just the name of the package we're
interested in, while ``xargs`` packs it nicely into a horizontal list, suitable
for feeding to ``pip install``. This is something you can stuff into your
``.bashrc``, and then call at your leisure.

## Updating using cargo ##

Luckily, ``cargo`` is not as elaborate as ``pip``, although it's far from
straightforward there either. Firstly, you're going to need
``cargo-update`` installed. Then, you have to do the only semi-obvious: 

```
cargo install-update --all
```

which will do exactly what you need. While you _could_ write a function like for
Python above, I find it's not really needed.

## Why this matters ##

A lot of people are going to start yelling something to the tune of "Pip and
Cargo aren't package managers!" right about now as a justification for not
including this functionality directly. I don't buy this excuse for
even one minute. Many Python-based and Rust-based executables give you
no other choice to install them - if your distro doesn't package them, then
you've got no other options. For _at least_ ``proselint``, I know my distro
doesn't, and I _dare_ you to try and find a distro that packages Tectonic!
Whether you like it or not, ``pip`` and ``cargo`` _are_ used as package
managers, and thus, easy updates are a requirement. Having to install other
packages just to make this reasonably smooth is not a reasonable thing to expect
on this basis.

Furthermore, these suggestions don't come up in searches - the edit history of
this page should show you that even for someone reasonably literate, this isn't
an easy thing to figure out! Why ``pip-autoremove`` and ``cargo-update``
_aren't_ included in ``pip`` and ``cargo`` by default is strange in itself, but
finding the information necessary to install and use them is also needlessly
hard in my opinion.

## What I'd like to see ##

``pip`` and ``cargo`` need to incorporate the ability to do one-command updates,
just like ``gem`` currently does:

```
[koz@Sebastian ~]$ gem update
Updating installed gems
Nothing to update
```

This doesn't appear hard relative what ``pip`` and ``cargo`` can _already_ do.
Not having this built-in and easily-findable in the documentation for either of
them is quite sad in my opinion, and should be fixed (but probably won't be).

[1]: https://www.gentoo.org/
[2]: http://tectonic-typesetting.github.io/en-US/
[3]: https://blog.getpelican.com/
[4]: https://rg3.github.io/youtube-dl/
[5]: http://asciidoctor.org/
[6]: https://github.com/amperser/proselint
[7]: https://github.com/mmstick/parallel
